Metrics for tracking performance of self-supervised training. It aims to give an idea about the quality of the learned representations during training in the presence of a labeled validation set. It can be used to decide how much longer to keep training. Since self-supervised models usually favor more epochs it can help save time and computation.

class KNNProxyMetric[source]

KNNProxyMetric(after_create=None, before_fit=None, before_epoch=None, before_train=None, before_batch=None, after_pred=None, after_loss=None, before_backward=None, before_step=None, after_cancel_step=None, after_step=None, after_cancel_batch=None, after_batch=None, after_cancel_train=None, after_train=None, before_validate=None, after_cancel_validate=None, after_validate=None, after_cancel_epoch=None, after_epoch=None, after_cancel_fit=None, after_fit=None) :: Callback

A metric which calculates knn-1 accuracy. Use with a labeled validation set.

Example Usage 

from self_supervised.layers import *
from self_supervised.vision.simclr import *

Create your dataset as usual. Make sure your validation set has labels. For example, if your dataset has 10% of labeled data and 90% of unlabeled data you can use that 10% or portion of it as your validation set. You assign dummy labels to 90% of data which doesn't have an actual label to circumvent code breaks during dls construction.

path = untar_data(URLs.MNIST_TINY)
items = get_image_files(path)
tds = Datasets(items, [PILImageBW.create, [parent_label, Categorize()]], splits=GrandparentSplitter()(items))
dls = tds.dataloaders(bs=5, after_item=[ToTensor(), IntToFloatTensor()], device='cpu')

Create your model and augmentations as usual

fastai_encoder = create_encoder('xresnet18', n_in=1, pretrained=False)
model = create_simclr_model(fastai_encoder, hidden_size=2048, projection_size=128)
aug_pipelines = get_simclr_aug_pipelines(size=28, rotate=False, jitter=False, bw=False, blur=False, stats=None, cuda=False)

Define self-supervised alogrithm in cbs. ShortEpochCallback is used for testing purposes here.

cbs=[SimCLR(aug_pipelines, temp=0.07, print_augs=True)]

Pipeline: RandomResizedCrop -> RandomHorizontalFlip
Pipeline: RandomResizedCrop -> RandomHorizontalFlip

We will use ValueMetric from fastai since KNNProxyMetric is a metric implemented as a Callback. ValueMetric expects a function which will return a value when it's called and that function is KNNProxyMetric.accuracy.

knn_metric_cb = KNNProxyMetric()
cbs += [knn_metric_cb]
metric = ValueMetric(knn_metric_cb.accuracy, metric_name='knn_accuracy')

Construct Learner with previously defined dls, cbs and metric.

learn = Learner(dls, model, cbs=cbs, metrics=metric)

learn.validate()

(#2) [2.2255146503448486,0.9628040194511414]